More endearing robot, method of controlling the same, and non-transitory recording medium

ABSTRACT

A more endearing robot includes an operation unit that causes the robot to operate, a viewing direction determiner that determines whether a viewing direction of a predetermined target is toward the robot or not, and an operation controller that controls the operation unit based on a result of determination by the viewing direction determiner.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromthe prior Japanese Patent Application No. 2017-122873, filed on Jun. 23,2017, the entire contents of which are incorporated herein by reference.

FIELD

This application relates to a more endearing robot, a method ofcontrolling the robot, and a non-transitory recording medium.

BACKGROUND

Robots that communicate with humans are known. For example, UnexaminedJapanese Patent Application Kokai Publication No. 2012-239557 disclosesa robot that serves as a pet-like toy having a shape representing ananimal. The robot disclosed in Unexamined Japanese Patent ApplicationKokai Publication No. 2012-239557 includes a liquid crystal display anda touch sensor at a position of the face. A petting operation on a faceimage displayed on the liquid crystal display varies the expression ofthe face image and causes a head and ears to sway from side to side.

SUMMARY

A robot according to a first aspect of the disclosure includes: anoperation unit that causes the robot to operate; a viewing directiondeterminer that determines whether a viewing direction of apredetermined target is toward the robot or not; and an operationcontroller that controls the operation unit based on a result ofdetermination by the viewing direction determiner.

A method of controlling a robot according to a second aspect of thedisclosure includes: causing the robot to operate; determining whether aviewing direction of a predetermined target is toward the robot or not;and controlling operation of the robot based on a result ofdetermination in the determining.

A non-transitory recording medium according to a third aspect of thedisclosure stores a program thereon. The program causes a computer of arobot to function as: an operation unit that causes the robot tooperate; a viewing direction determiner that determines whether theviewing direction of a predetermined target is toward the robot or not;and an operation controller that controls the operation unit based on aresult of determination by the viewing direction determiner.

Additional objects and advantages of the invention will be set forth inthe description which follows, and in part will be obvious from thedescription, or may be learned by practice of the invention. The objectsand advantages of the invention may be realized and obtained by means ofthe instrumentalities and combinations particularly pointed outhereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention, andtogether with the general description given above and the detaileddescription of the embodiments given below, serve to explain theprinciples of the invention.

FIG. 1 is a front view of a robot according to an embodiment of thedisclosure;

FIG. 2 is a perspective view of the robot according to the embodiment ofthe disclosure;

FIG. 3 is a block diagram illustrating the hardware configuration of therobot according to the embodiment of the disclosure;

FIG. 4 is a block diagram illustrating the functional configuration ofthe robot according to the embodiment of the disclosure;

FIG. 5 is a first table illustrating exemplary solo operations executedby the robot according to the embodiment of the disclosure;

FIG. 6 is a second table illustrating the exemplary solo operationsexecuted by the robot according to the embodiment of the disclosure;

FIGS. 7A to 7C each illustrate an exemplary captured image according tothe embodiment of the disclosure;

FIGS. 8A to 8C each illustrate an exemplary movement of a movablecomponent of the robot according to the embodiment of the disclosure;

FIGS. 9A to 9C each illustrate an exemplary change in images displayedat eyes of the robot according to the embodiment of the disclosure;

FIG. 10 is a flowchart illustrating a robot controlling process executedby the robot according to the embodiment of the disclosure; and

FIG. 11 is a flowchart illustrating a viewing direction determiningprocess executed by the robot according to the embodiment of thedisclosure.

DETAILED DESCRIPTION

Embodiments of the disclosure will now be described with reference tothe accompanying drawings. The identical or corresponding components areprovided with the same reference character in the drawings.

FIGS. 1 and 2 illustrate the appearance of a robot 100 according to anembodiment of the disclosure. The robot 100 is a pet robot similar to apet. The robot 100 has a shape representing an animal and autonomouslyoperates in accordance with a predetermined operational program.

The robot 100 executes various operations in response to externalstimuli, such as a call and contact from a predetermined target thatexists outside of the robot 100. The robot 100 can thus communicate andinteract with the predetermined target. The predetermined target existsoutside of the robot 100 and communicates and interacts with the robot100. Specific examples of the predetermined target include a user whoowns the robot 100, people around the user (for example, family membersand friends of the user), animals around the user (for example, petskept by the user or other people), and robots other than the robot 100.The predetermined target can also be called communication target,communication partner, interaction target, or interaction partner, forexample.

With reference to FIGS. 1 and 2, the robot 100 has a three-dimensionalshape that represents a small dog in appearance. The robot 100 is mainlycomposed of, for example, a hard synthetic resin, such as plastic. Therobot 100 includes a head 101, a body 102, ears 103, eyes 104, a mouth105, arms 107, legs 108, and a tail 109.

The head 101, the ears 103, the arms 107, the legs 108, and the tail 109can be moved by driving members installed in the robot 100. The head 101is attached to the body 102 with a neck joint provided at the positionof the neck, such that the head 101 can rotate in the three directions(pitch, roll, and yaw directions). Each of the eyes 104 is provided witha display 117 that displays an eye image (for example, an eyeballimage). The mouth 105 is provided with an imager 115 a that captures animage of an object in front of the robot 100.

FIG. 3 illustrates hardware configuration of the robot 100. Withreference to FIG. 3, the robot 100 includes a central processing unit(CPU) 110, a clock 110 a, a memory 111, a battery 112, a driver 113,movable components 114, a sensor unit 115, a wireless communicator 116,displays 117, a sound output device 118, and an image recognizer 119.

The CPU 110 is the central processing unit, such as a microprocessor,that executes various processes and calculations. The CPU 110 isconnected to individual components of the robot 100 via system busesserving as transmission paths for transferring instructions and data andthus controls the entire robot 100. The clock 110 a includes a real timeclock (RTC).

The memory 111 includes a random access memory (RAM) functioning as aworking memory of the CPU 110, a read-only memory (ROM), and anon-volatile memory, such as a flash memory. The memory 111 storesprograms and data for execution of various processes by the CPU 110, forexample, an operating system (OS) and application programs. The memory111 also stores data generated or acquired through the various processesof the CPU 110.

The battery 112 is a rechargeable battery that stores electrical energyand supplies electrical power to individual components of the robot 100.The battery 112 is recharged by a charging station when the robot 100 isreturned to the charging station.

The driver 113 includes driving members, such as motors and actuatorsfor driving the movable components 114 of the robot 100, and a drivecircuit for driving the driving members. The movable components 114 canmove in certain manners. Specifically, the movable components 114include the head 101, the ears 103, the arms 107, the legs 108, and thetail 109. The CPU 110 transmits a control signal to the drive circuitbased on the operational program. In accordance with the control signaltransmitted from the CPU 110, the drive circuit supplies the drivingmembers with a pulse signal for driving the driving members. Inaccordance with the pulse signal supplied from the drive circuit, thedriving members drive the movable components 114.

The driving of the movable components 114 by the driver 113 can achievevarious operations of the robot 100. For example, movements of the arms107 and the legs 108 enable the robot 100 to move forward or rearward orturn to another direction. In addition, movements of the head 101,movements of the ears 103, and wagging of the tail 109, for example,enable the robot 100 to operate and behave like a real dog.

The sensor unit 115 includes a plurality of sensors for detectingphysical quantities inside or around the robot 100. With reference toFIG. 3, the sensor unit 115 includes the imager 115 a that captures animage representing the circumference of the robot 100, sound sensors 115b that detect a sound, a contact sensor 115 c that detects the contactwith the robot 100, and a distance sensor 115 d that measures a distanceto a surrounding object. The sensor unit 115 further includes anacceleration sensor that detects a movement of the robot 100, a gyrosensor that detects a rotation of the robot 100, a geomagnetic sensorthat determines an orientation of the robot 100, a temperature sensorthat determines an temperature around the robot 100, and an atmosphericpressure sensor that determines an atmospheric pressure around the robot100 (which are not shown).

The imager 115 a is a so-called camera and provided to the mouth 105.The imager 115 a includes an image capturer that collects lightreflected from a subject and captures an image of the subject, and animage processor that processes the image captured by the image capturer.The sound sensors 115 b are provided to the head 101 and detect voicesfrom the predetermined target and environmental sounds, for example. Therobot 100 includes a plurality of microphones (not shown) along thesurface of the head 101 that function as the sound sensors 115 b. Thesemicrophones can efficiently detect sounds occurring around the robot100. The other sensors are provided to certain components of the robot100 and acquire information indicating a state inside or around therobot 100. These sensors thus enable the sensor unit 115 to acquire theinformation indicating the state inside or around the robot 100 andprovide the information to the CPU 110.

The wireless communicator 116 includes an interface for wirelesscommunication with external apparatuses. The wireless communicator 116executes the wireless communication with the charging station under thecontrol of the CPU 110 in accordance with a communication standard, suchas a wireless local area network (LAN) standard (for example, a wirelessfidelity (Wi-Fi) standard) or a Bluetooth (registered trademark)standard.

Each of the displays 117 is, for example, a liquid crystal display, anorganic electroluminescence (EL) display, or a light emitting diode(LED) display. The displays 117 are provided to the positions of therespective eyeballs of the eyes 104 and display various imagesappropriate for the situations under the control of a display drivingcircuit (not shown).

The sound output device 118 includes a speaker and a sound outputinterface. The sound output device 118 converts audio data generated bythe CPU 110 into the sound and outputs the sound to the outside. Thespeaker is provided to the head 101. The sound output device 118 outputsvarious sounds including an animal cry and a human language. Forexample, the robot 100 collects a speech of the predetermined targetwith the sound sensors 115 b and then outputs the sound corresponding tocontents of the speech of the predetermined target through the soundoutput device 118. The robot 100 can thus have a simple conversationwith the predetermined target.

The image recognizer 119 includes an image processor, such as a digitalsignal processor (DSP) or a graphics processing unit (GPU), and a buffermemory that temporarily stores the image to be processed. The imagerecognizer 119 analyzes the image captured by the imager 115 a andrecognizes, for example, a person, a face, an object, or a patterncontained in the image captured by the imager 115 a based on well-knownimage recognition techniques.

The functional configuration of the robot 100 will now be described withreference to FIG. 4. The robot 100 includes a target detector 120, anaction detector 130, an operation controller 140, and a viewingdirection determiner 150 in terms of function, as illustrated in FIG. 4.The CPU 110 reads programs stored in the ROM, loads the programs intothe RAM, and executes and controls the programs to function as theabove-mentioned components.

The target detector 120 detects the predetermined target that existsoutside of the robot 100. The predetermined target indicates theinteraction partner (the communication partner) that interacts with therobot 100, for example, the user who owns the robot 100, the personaround the user, or the animal.

The target detector 120 captures the image representing thecircumference of the robot 100 with the imager 115 a provided to themouth 105. The target detector 120 then analyzes the image captured bythe imager 115 a with the image recognizer 119 to determine whether anyhuman or animal is contained in the image or not. The target detector120 can thus be achieved by cooperation of the CPU 110 with the imager115 a and the image recognizer 119.

The action detector 130 detects an action of the predetermined target tothe robot 100. The action of the predetermined target to the robot 100indicates the action, such as a talk or contact, of the predeterminedtarget for a purpose of interaction (communication) with the robot 100.Examples of the action of the predetermined target to the robot 100include calling the robot 100, touching the surface of the robot 100,and gesturing in front of the robot 100. The action detector 130 detectssuch actions of the predetermined target with the individual sensors ofthe sensor unit 115.

Specifically, in a case in which the predetermined target calls therobot 100, the action detector 130 detects a voice from thepredetermined target with the sound sensors 115 b. In a case in whichthe predetermined target touches the robot 100, the action detector 130detects the contact with the contact sensor 115 c. Furthermore, in acase in which the predetermined target gestures in front of the robot100, the action detector 130 detects a gesture with the imager 115 a.The action detector 130 can thus be achieved by the cooperation of theCPU 110 with the individual sensors of the sensor unit 115.

The operation controller 140 controls the operation unit 170 and causesthe operation unit 170 to execute operations defined in an operationtable 190. The operation unit 170 includes the movable components 114,the displays 117, and the sound output device 118. The operation unit170 causes the robot 100 to operate by moving the movable components114, displaying images on the displays 117, or outputting the soundthrough the sound output device 118. The operation table 190 thatdefines operations of the operation unit 170 is preliminarily stored inthe memory 111. The operation controller 140 causes the operation unit170 to execute various operations appropriate for the situations withreference to the operation table 190. The operation controller 140 canthus be achieved by the cooperation of the CPU 110 with the driver 113,the movable components 114, the displays 117, and the sound outputdevice 118.

In a case in which the action detector 130 detects any action of thepredetermined target to the robot 100, the operation controller 140causes the operation unit 170 to execute the operation for responding tothe detected action. Such the operation is provided for the purpose ofinteraction (communication) with the predetermined target and is alsocalled an interacting operation (an interacting action) or a respondingoperation (a responding action).

Specifically, in a case in which the action detector 130 detects anyspeech from the predetermined target, the operation controller 140causes the sound output device 118 to output the sound corresponding tothe detected speech, turns the head 101 to the predetermined target, ormoves the arms 107 and legs 108 to approach the predetermined target. Ina case in which the action detector 130 detects any contact, theoperation controller 140 wags the tail 109 or displays certain images onthe displays 117. Furthermore, in a case in which the action detector130 detects any gesture of the predetermined target, the operationcontroller 140 causes the sound output device 118 to output the soundcorresponding to the detected gesture or displays the certain images onthe displays 117.

As described above, the operation controller 140 causes the robot 100 toexecute various interacting operations in response to different actionsof the predetermined target detected by the action detector 130. Thepredetermined target can thus enjoy communicating with the robot 100.The operation table 190 defines the correspondence relationship betweenactions of the predetermined target and interacting operations to beexecuted by the robot 100 in a case in which the action detector 130detects the respective actions. The operation controller 140 refers tothe operation table 190 and determines the interacting operation to beexecuted by the operation unit 170.

In contrast, in a case in which the action detector 130 detects noaction of the predetermined target to the robot 100, the operationcontroller 140 causes the robot 100 to execute a solo operationdifferent from the interacting operation. The solo operation indicatesan operation that the robot 100 executes alone, independently from thepredetermined target, which is a spontaneous operation withoutinteraction (communication) with the predetermined target. The solooperation is also called solo action or solo play.

In other words, in a case in which the robot 100 is going to interactwith any target that exists around the robot 100, the operationcontroller 140 causes the robot 100 to execute the above-describedinteracting operation. In contrast, in a case in which the robot 100 isnot interacting with any target that exists around the robot 100, theoperation controller 140 causes the robot 100 to execute the solooperation. The robot 100 can thus naturally behave like a real pet andcreate a more endearing impression.

FIGS. 5 and 6 illustrate exemplary solo operations defined in theoperation table 190. With reference to FIGS. 5 and 6, the operationtable 190 defines the solo operation to be executed by the robot 100 ineach condition.

FIG. 5 illustrates the exemplary solo operations in a case where thepredetermined target exists around the robot 100 but does not interactwith the robot 100. After elapse of a predetermined interval from thelast detection of the action of the predetermined target to the robot100 by the action detector 130, the operation controller 140 causes therobot 100 to execute any of the solo operations listed in FIG. 5.Specifically, the operation controller 140 causes the robot 100 toexecute the solo operation, which varies depending on information onemotions of the robot 100. The information on the emotions of the robot100 indicates information representing feelings (for example, delight,anger, sorrow, and pleasure) of the robot 100, which is determined tosimulate the real animal.

Specifically, as illustrated in FIG. 5, the information on the emotionsof the robot 100 indicates any of four emotions: “lonely”, “motivated”,“happy”, and “peaceful”, which are defined based on a first emotionalvalue “calm” representing a level of calmness and a second emotionalvalue “excite” representing a level of excitement. The operationcontroller 140 varies the first and the second emotional values and thuschanges the emotions of the robot 100 in response to the actions of thepredetermined target during interaction between the robot 100 and thepredetermined target and depending on other situations.

For example, in a case in which the robot 100 expresses the “lonely”feeling, the operation controller 140 causes the robot 100 to executeany of the solo operations including “Look for something, move to acorner of a room, and gaze”. In a case in which the robot 100 expressesthe “motivated” feeling, the operation controller 140 causes the robot100 to execute any of the solo operations including “Move to a corner ofa room and wander around to show a desire to go outside”. In a case inwhich the robot 100 expresses the “happy” feeling, the operationcontroller 140 causes the robot 100 to execute any of the solooperations including “Sing”. In a case in which the robot 100 expressesthe “peaceful” feeling, the operation controller 140 causes the robot100 to execute any of the solo operations including “Gaze out a window”.The solo operation to be executed is selected at random from among thesolo operations executable for each emotion.

FIG. 6 illustrates example of solo operations induced by a surroundingenvironment of the robot 100 or time. In a case in which the actiondetector 130 detects no action of the predetermined target to the robot100 and in a case in which at least one of the surrounding environmentof the robot 100 and the time satisfies a specified condition, theoperation controller 140 causes the robot 100 to execute correspondingone of the solo operations listed in FIG. 6.

The surrounding environment of the robot 100 indicates information onthe outside of the robot 100 except for the predetermined target thatcan interact with the robot 100, for example, sound volume, temperature,atmospheric pressure, brightness, and existence of obstacle. Thesurrounding environment is detected by, for example, the sound sensors115 b, the contact sensor 115 c, the distance sensor 115 d, thetemperature sensor, and the atmospheric pressure sensor. The timeindicates information on current time, date, or season, for example, andis measured by the clock 110 a. The specified condition indicates acondition related to the surrounding environment or the time. As listedin FIG. 6, examples of the specified condition include conditionsrelated to the surrounding environment, such as “Hearing music” and“Hearing a conversation of people”, and conditions related to the time,such as “At a bed time” and “On a new year morning or a birthdaymorning”.

In a case in which at least one of the surrounding environment of therobot 100 and the time satisfies the specified condition, the operationcontroller 140 causes the robot 100 to execute the solo operationcorresponding to the satisfied condition. For example, in a case inwhich the condition of “Hearing music” is satisfied, the operationcontroller 140 causes the robot 100 to execute the solo operation of“Change a facial expression and wander around”. In a case in which thecondition of “Being petted before recognizing a person” is satisfied,the operation controller 140 causes the robot 100 to execute the solooperation of “Get surprised and look around”. As described above, inresponse to an external stimulus, such as the surrounding environment orthe time, the operation controller 140 causes the robot 100 to executethe solo operation corresponding to the stimulus by operating at leastone of the movable components 114, the displays 117, and the soundoutput device 118 included in the operation unit 170.

The description of the functional configuration of the robot 100illustrated in FIG. 4 will be resumed. The viewing direction determiner150 determines whether the viewing direction of the predetermined targetis toward the robot 100 during execution of the solo operation of therobot 100 or not. To determine whether the viewing direction of thepredetermined target is toward the robot 100 or not, the viewingdirection determiner 150 executes three processes: (1) detection of aface, (2) determination of an orientation of the face, and (3) detectionof the viewing direction.

First, the viewing direction determiner 150 detects the face from theimage captured by the imager 115 a. FIGS. 7A to 7C each illustrate anexemplary captured image 200 containing a user 210 (a predeterminedtarget). FIG. 7A illustrates the image 200 of the user 210 captured fromthe front, and FIGS. 7B and 7C each illustrate the image 200 of the user210 captured in an oblique direction. The viewing direction determiner150 detects a face area 220 corresponding to the face of the user 210with the image recognizer 119 by a well-known face recognitiontechnique, for example, from the captured image 200 illustrated in anyof FIGS. 7A to 7C.

Specifically, the viewing direction determiner 150 specifies pixelshaving a skin color from the pixels contained in the captured image 200and determines an area of the specified pixels having the skin color tobe the face area 220. Alternatively, the viewing direction determiner150 may extract facial parts (for example, eyes, a nose, and a mouth) ascharacteristic points from the captured image 200 and determine the facearea 220 based on positions of the extracted facial parts. The viewingdirection determiner 150 thus detects the face of the user 210 from thecaptured image 200.

Second, the viewing direction determiner 150 determines an orientationof the face detected from the captured image 200 by a well-knowntechnique. Specifically, the viewing direction determiner 150 determinesthe orientation of the face in three directions (pitch, roll, and yawdirections) based on the positional relationship between the facialparts (for example, the eyes, the nose, and the mouth) in the determinedface area 220.

For example, in the captured image 200 illustrated in FIG. 7A, the faceof the user 210 is directed to the front, that is, directed to the robot100. In this case, the viewing direction determiner 150 determines theorientation of the face in each of the pitch, roll, and yaw directionsto be 0°. In contrast, in the captured image 200 illustrated in FIG. 7Bor 7C, the face of the user 210 is inclined in the yaw direction whilenot inclined in the pitch or roll direction. In this case, the viewingdirection determiner 150 determines the orientation of the face to beinclined by −15° in the yaw direction, for example. The viewingdirection determiner 150 thus determines the orientation of the face ofthe user 210 in the captured image 200.

Third, the viewing direction determiner 150 detects the viewingdirection of the user 210 in the captured image 200 by a well-knowntechnique. Specifically, the viewing direction determiner 150 specifiesthe eyes contained in the face area 220 and detects the position ofirises (pupils) in the respective specified eyes. The viewing directiondeterminer 150 then determines the viewing direction of the user 210,that is, the direction of view of the user 210 based on the positions ofthe irises in the eyes.

On the basis of the above-described three processes: (1) detection ofthe face, (2) determination of the orientation of the face, and (3)detection of the viewing direction, the viewing direction determiner 150determines whether the viewing direction of the predetermined target istoward the robot 100 or not. For example, in a case in which thedetected face is located in a center of the captured image 200 and in acase in which the face and the viewing direction are toward the front,the viewing direction determiner 150 determines that the viewingdirection of the predetermined target is toward the robot 100. Inaddition, in a case in which the detected face is located in a positionother than the center of the captured image 200 and in a case in whichthe face or the viewing direction is toward the imager 115 a, theviewing direction determiner 150 also determines that the viewingdirection of the predetermined target is toward the robot 100. Theviewing direction determiner 150 thus determines whether the viewingdirection of the predetermined target is toward the robot 100 or notbased on the image of the predetermined target captured by the imager115 a.

In a case of determining that the viewing direction of the predeterminedtarget is toward the robot 100 during the execution of the solooperation by the robot 100, the viewing direction determiner 150 startsmeasuring time with the clock 110 a. The viewing direction determiner150 then determines whether the viewing direction of the predeterminedtarget is toward the robot 100 at least for a predetermined duration ornot. The predetermined duration is, for example, a period approximatelyfrom one to several seconds. The predetermined duration is set andstored in the memory 111 in advance. The viewing direction determiner150 can thus be achieved by the cooperation of the CPU 110 with theimager 115 a, the image recognizer 119, and the clock 110 a.

In contrast, the viewing direction determiner 150 does not determinewhether the viewing direction of the predetermined target is toward therobot 100 or not while the robot 100 is halting the solo operation. Inother words, during execution of the interacting operation by the robot100 with the predetermined target, the interacting operation isprioritized over the determination on the viewing direction of thepredetermined target, and the viewing direction determiner 150 thus doesnot execute the determination.

The operation controller 140 controls the operation unit 170 based on aresult of determination by the viewing direction determiner 150.Specifically, in a case in which the viewing direction determiner 150determines that the viewing direction of the predetermined target istoward the robot 100 at least for the predetermined duration, theoperation controller 140 causes the operation unit 170 to execute afirst operation for responding to the viewing direction.

The first operation indicates an operation for responding to the viewingdirection of the predetermined target toward the robot 100 and ispreliminarily defined as a part of the operation table 190. Specificexamples of the first operation include (1) moving the movablecomponents 114, (2) changing the images displayed on the displays 117,and (3) outputting the sound through the sound output device 118. In acase in which the viewing direction determiner 150 determines that theviewing direction of the predetermined target is toward the robot 100 atleast for the predetermined duration, the operation controller 140refers to the operation table 190 and causes the operation unit 170 toexecute at least one of the operations defined in the operation table190 as the first operation.

(1) First, the operation controller 140 causes the operation unit 170 toexecute the first operation by moving the movable components 114.Specifically, the operation controller 140 causes the driver 113 todrive the neck joint to move the head 101. For example, as illustratedin FIG. 8A, the operation controller 140 may rotate and incline the head101 in the roll direction in the first operation. Alternatively, asillustrated in FIG. 8B, the operation controller 140 may rotate and nodthe head 101 in the pitch direction. Alternatively, the operationcontroller 140 may cause the head 101 to face the predetermined target.Alternatively, as illustrated in FIG. 8C, the operation controller 140may wag the tail 109 for the predetermined target in the firstoperation.

(2) Second, the operation controller 140 causes the operation unit 170to execute the first operation by changing the images displayed on thedisplays 117. Specifically, the operation controller 140 may change theimages of the irises (pupils) displayed on the displays 117, forexample, into images containing larger pupils as illustrated in FIG. 9A,into images of closed eyes as illustrated in FIG. 9B, or into images ofdisplaced pupils as illustrated in FIG. 9C. Alternatively, the operationcontroller 140 may cause other images to be displayed on the displays117 such that the robot 100 has tears in the eyes, closes one of theeyes (winks), looks at the predetermined target (makes eye contact), orquickly closes and opens the eyes (blinks).

(3) Third, the operation controller 140 causes the operation unit 170 toexecute the first operation by outputting the sound through the soundoutput device 118. Specifically, the operation controller 140 may causethe sound output device 118 to output the sound, such as a spokenlanguage “What!” or “You looked at me!”, to the predetermined target asa response to the viewing direction of the predetermined target. Thesound to be output is selected from preset candidates in accordance witha predetermined rule.

In a case in which the viewing direction of the predetermined target istoward the robot 100 at least for the predetermined duration, theoperation controller 140 causes the operation unit 170 to execute atleast one of these first operations. Specifically, after the recognitionof the face of the predetermined target by the image recognizer 119, theoperation controller 140 causes the operation unit 170 to execute thefirst operation, which varies depending on a result of recognition bythe image recognizer 119. In other words, the operation controller 140determines parameters, such as age, sex, and facial expression, of thepredetermined target based on the face recognized by the imagerecognizer 119 and then determines whether to (1) move the movablecomponents 114, (2) change the images displayed on the displays 117, (3)output the sound through the sound output device 118, or execute acombination of these operations, depending on a result of determinationon the parameters. The robot 100 can thus provide various responses fordifferent interaction targets and create a more endearing impression.

Until the elapse of the predetermined duration from when the viewingdirection of the predetermined target is turned to the robot 100, theoperation controller 140 causes the operation unit 170 to execute thesolo operation. In other words, regardless of determination that theviewing direction of the predetermined target is toward the robot 100 bythe viewing direction determiner 150, the operation controller 140causes the operation unit 170 to continue the ongoing solo operationwithout an operation for responding to the viewing direction until theelapse of the predetermined duration. The operation controller 140 thusenables the robot 100 to pretend not to notice the predetermined targetuntil the elapse of the predetermined duration from when the viewingdirection of the predetermined target is turned to the robot 100.

In a case in which the viewing direction determiner 150 determines thatthe viewing direction of the predetermined target is not toward therobot 100, the operation controller 140 causes the operation unit 170 toexecute a second operation different from the first operation. Thesecond operation indicates an operation intended to prompt thepredetermined target to have the viewing direction toward the robot 100and is preliminarily defined as a part of the operation table 190. Forexample, the operation controller 140 may rotate the head 101 to attractthe attention of the predetermined target. Alternatively, the operationcontroller 140 may drive the legs 108 so that the robot 100 approachesthe predetermined target or moves to be located in the viewing directionof the predetermined target.

In a case in which such the second operation succeeds in causing thepredetermined target to have the viewing direction toward the robot 100,the operation controller 140 causes the operation unit 170 to start theabove-described first operation. In a case in which the predeterminedtarget takes an action, such as a conversation or contact, directed tothe robot 100, the operation controller 140 causes the operation unit170 to execute the interacting operation with the predetermined target.The operation controller 140 thus executes an operation for attractingthe attention of the predetermined target in a case in which thepredetermined target does not look at the robot 100 during execution ofthe solo operation by the robot 100. This configuration can make therobot 100 more endearing.

The process executed by the robot 100 having the above-describedconfiguration will now be explained with reference to flowcharts ofFIGS. 10 and 11.

A robot controlling process executed by the robot 100 will now beexplained with reference to the flowchart of FIG. 10. The robotcontrolling process illustrated in FIG. 10 is started when activation ofthe robot 100 and recharge of the battery 112 turn the robot 100 into anormally operable mode.

At a start of the robot controlling process, the CPU 110 determineswhether any predetermined target is detected (Step S1) or not.Specifically, the CPU 110 causes the imager 115 a to capture the imageof the circumference of the robot 100. The CPU 110 then analyzes thecaptured image and determines whether any predetermined target, such asa human or an animal, as an interaction (communication) partner with therobot 100 exists around the robot 100 or not. The CPU 110 thus functionsas the target detector 120 in Step S1.

In a case in which no predetermined target as the interaction partner isdetected in the determination (Step S1; NO), the CPU 110 remains in StepS1 and waits until detection of any predetermined target.

In contrast, in a case in which any predetermined target as theinteraction partner is detected (Step S1; YES), the CPU 110 determineswhether to interact with the predetermined target (Step S2) or not.Specifically, the CPU 110 determines whether the sound sensors 115 b,the contact sensor 115 c, or the imager 115 a detects the call, thecontact, or the gesture of the predetermined target to the robot 100,for example. In a case in which any action of the predetermined targetto the robot 100 is detected, the CPU 110 determines that the robot 100is to interact with the predetermined target. The CPU 110 thus functionsas the action detector 130 in Step S2.

In a case in which determining that the robot 100 is to interact withthe predetermined target (Step S2; YES), the CPU 110 causes the robot100 to execute the interacting operation (Step S3). The interactingoperation indicates the operation for interacting with the predeterminedtarget and responding to the action of the predetermined target detectedin Step S2. Specifically, the CPU 110 controls the movable components114, the displays 117, and the sound output device 118 included in theoperation unit 170 and thus causes the operation unit 170 to execute theoperation corresponding to the detected action. This process enables therobot 100 to interact (communicate) with the predetermined target. TheCPU 110 thus functions as the operation controller 140 in Step S3.

In contrast, in a case in which determining that the robot 100 is not tointeract with the predetermined target (Step S2; NO), the CPU 110 skipsStep S3.

After the interaction with the predetermined target or after skippingStep S3, the CPU 110 determines whether the solo operation condition issatisfied (Step S4) or not. The solo operation condition indicates thecondition required for execution of the solo operation by the robot 100.Specifically, the solo operation condition is satisfied in a case inwhich the predetermined interval has elapsed from the last detection ofthe action of the predetermined target to the robot 100 in Step S2, orin a case in which at least one of the surrounding environment of therobot 100 and the time satisfies any of the specified conditions listedin FIG. 6. The CPU 110 determines whether the solo operation conditionis satisfied or not based on the detection of the surroundingenvironment by the sensor unit 115 and the time measurement by the clock110 a. The CPU 110 thus functions as a solo operation determiner fordetermining whether the robot 100 is executing the solo operation.

In a case in which the solo operation condition is not satisfied (StepS4; NO), the CPU 110 returns to Step S1 and determines again whether anypredetermined target is detected or not. While the predetermined targetis being detected with satisfaction of the condition required for theinteraction with the predetermined target, the CPU 110 repeats theprocess from Steps S1 to S4 to continue the interaction with thepredetermined target.

In contrast, in a case in which the solo operation condition issatisfied (Step S4; YES), the CPU 110 causes the robot 100 to executethe solo operation (Step S5). Specifically, the CPU 110 controls themovable components 114, the displays 117, and the sound output device118 included in the operation unit 170 and thus causes the operationunit 170 to execute the solo operation corresponding to the satisfiedcondition.

For example, in a case in which the predetermined interval has elapsedfrom the last detection of the action of the predetermined target, theCPU 110 causes the operation unit 170 to execute any of the solooperations listed in FIG. 5 depending on information on the emotions ofthe robot 100 at that time. In contrast, in a case in which at least oneof the surrounding environment of the robot 100 and the time satisfiesthe specified condition, the CPU 110 causes the operation unit 170 toexecute any of the solo operations listed in FIG. 6. The CPU 110 thusfunctions as the operation controller 140 in Step S5.

During execution of the solo operation by the robot 100, the CPU 110executes a viewing direction determining process (Step S6). The viewingdirection determining process in Step S6 will now be explained in detailwith reference to the flowchart of FIG. 11.

At a start of the viewing direction determining process illustrated inFIG. 11, the CPU 110 detects the face from the image captured by theimager 115 a (Step S61). Specifically, the CPU 110 determines the facearea 220 corresponding to the face of the predetermined target in thecaptured image 200 illustrated in any of FIGS. 7A to 7C, for example, bythe well-known face recognition technique.

After detection of the face from the captured image, the CPU 110determines the orientation of the face (Step S62). Specifically, the CPU110 determines the orientation of the face in the three directions (thepitch, the roll, and the yaw directions) based on the positionalrelationship between the facial parts (for example, the eyes, the nose,and the mouth) in the detected face area 220.

After determination of the orientation of the face, the CPU 110 detectsthe viewing direction of the predetermined target (Step S63).Specifically, the CPU 110 specifies the eyes contained in the face area220 and determines the direction of a view of the predetermined targetbased on the positions of the irises in the respective specified eyes.

After detection of the viewing direction, the CPU 110 determines whetherthe viewing direction of the predetermined target is toward the robot100 (Step S64) or not. Specifically, in a case in which the detectedface is located in the center of the captured image 200 and in a case inwhich the face and the viewing direction are toward the front, the CPU110 determines that the viewing direction of the predetermined target istoward the robot 100. In addition, in a case in which the detected faceis located in a position other than the center of the captured image 200and in a case in which the face or the viewing direction is toward theimager 115 a, the CPU 110 also determines that the viewing direction ofthe predetermined target is toward the robot 100.

In a case in which determining that the viewing direction of thepredetermined target is toward the robot 100 (Step S64; YES), the CPU110 causes the clock 110 a to start measuring the time and thendetermines whether the predetermined duration has elapsed from when theviewing direction of the predetermined target is turned to the robot 100(Step S65) or not.

In a case in which the predetermined duration has not elapsed (Step S65;NO), the CPU 110 causes the operation unit 170 to continue the solooperation (Step S66). The CPU 110 then returns to Step S64 anddetermines again whether the viewing direction of the predeterminedtarget is toward the robot 100 or not. In other words, even afterdetermination that the viewing direction of the predetermined target istoward the robot 100, the CPU 110 causes the robot 100 to continue theongoing solo operation and pretend to not to notice the viewingdirection until the elapse of the predetermined duration. The CPU 110then determines whether the viewing direction of the predeterminedtarget is toward the robot 100 at least for the predetermined durationor not.

In a case in which determining that the viewing direction of thepredetermined target is toward the robot 100 at least for thepredetermined duration (Step S65; YES), the CPU 110 causes the operationunit 170 to execute the first operation for responding to the viewingdirection (Step S67). Specifically, the CPU 110 causes the imagerecognizer 119 to recognize the face of the predetermined target. Basedon the result of face recognition, the CPU 110 causes the operation unit170 to execute one or more of the operations including: (1) moving themovable components 114, (2) changing the images displayed on thedisplays 117, and (3) outputting the sound through the sound outputdevice 118.

In contrast, in a case in which determining that the viewing directionof the predetermined target is not toward the robot 100 in Step S64(Step S64; NO), the CPU 110 causes the operation unit 170 to execute thesecond operation (Step S68). Specifically, the CPU 110 prompts thepredetermined target to have the viewing direction toward the robot 100by rotating the head 101 or displacing the robot 100.

The viewing direction determining process illustrated in FIG. 11 is thenterminated. In the viewing direction determining process illustrated inFIG. 11, the CPU 110 functions as the viewing direction determiner 150from Steps S61 to S65 and functions as the operation controller 140 fromSteps S66 to S68.

Referring back to FIG. 10, after the viewing direction determiningprocess, the CPU 110 returns to Step S1 and determines again whether anypredetermined target is detected or not. In a case in which anypredetermined target is detected in the determination, the CPU 110executes the process in Step S2 and the following steps. The CPU 110thus repeats the interacting operation with the predetermined target orthe solo operation while the predetermined target is being detected.During the solo operation, the CPU 110 executes the viewing directiondetermining process.

As explained above, the robot 100 according to the embodiment determineswhether the viewing direction of the predetermined target is toward therobot 100 during the solo operation or not and controls the operationunit 170 based on the result of determination on the viewing direction.The robot 100 can thus respond to the viewing direction and create themore endearing impression.

In particular, even if the viewing direction of the predetermined targetis toward the robot 100 according to the embodiment, the operation unit170 continues the solo operation until the elapse of the predeterminedduration. After the elapse of the predetermined duration from when theviewing direction is toward the robot, the operation unit 170 executesthe first operation for responding to the viewing direction. The robot100 thus pretends not to notice the viewing direction toward the robot100 for a while. Accordingly, the robot 100 can behave like the realanimal and create the further more endearing impression.

Modifications

The above-described embodiments of the disclosure are mere examples andshould not be construed as limiting the scope of the disclosure. Thatis, the embodiments of the disclosure may be modified in various mannersand these modified embodiments are still involved in the scope of thedisclosure.

For example, the robot 100 is the pet robot in the similitude of thesmall dog in the embodiments. Alternatively, the robot 100 according tothe disclosure may have a shape other than the shape that represents thesmall dog. For example, the robot 100 according to the disclosure may bea robot similar to a large dog, other animal, such as a cat, a mouse, ora rabbit, or a human.

Although the displays 117 are provided to the respective eyes 104 andthe imager 115 a is provided to the mouth 105 in the embodiments, thedisplays 117 may be provided to the components other than the eyes 104and the imager 115 a may be provided to the component other than themouth 105 (for example, a nose) according to the disclosure. In a casein which the displays 117 are not provided to the eyes 104, theoperation controller 140 controls the direction of the view of the robot100 by mechanically moving the eyeball components.

In a case in which the viewing direction of the predetermined target istoward the robot 100 at least for the predetermined duration, theoperation controller 140 causes the operation unit 170 to execute thefirst operation, which varies depending on the result of recognition bythe image recognizer 119 in the embodiments. Alternatively, theoperation controller 140 according to the disclosure may cause theoperation unit 170 to execute the first operation, which variesdepending on the distance to the predetermined target measured by thedistance sensor 115 d. For example, in a case in which the distance tothe predetermined target measured by the distance sensor 115 d isshorter than a predetermined distance, the operation controller 140 maychange the images displayed on the displays 117 in the first operation.In this case, the operation controller 140 may adjust the imagesdisplayed on the displays 117 such that the direction of view of therobot 100 is displaced from the eyes of the predetermined target, so asto reduce a feeling of pressure due to a short distance.

Alternatively, in a case in which the viewing direction of thepredetermined target is toward the robot 100 at least for thepredetermined duration, the operation controller 140 may cause theoperation unit 170 to execute the first operation, which variesdepending on whether the robot 100 looks at the predetermined target ornot. For example, in a case in which the robot 100 looks at thepredetermined target, the operation controller 140 may control themovable components 114, the displays 117, and the sound output device118, as illustrated in FIGS. 8A to 8C and FIGS. 9A to 9C. In a case inwhich the robot 100 does not look at the predetermined target, theoperation controller 140 may control the movement of the head 101 andthe display on the eyes 104 such that the robot 100 looks at thepredetermined target. Alternatively, the operation controller 140 maycause the operation unit 170 to execute the first operation, whichvaries depending on the ongoing solo operation.

In the robot 100 according to the embodiments, the CPU 110 executes theprogram stored in the ROM and thus functions as each of the targetdetector, the action detector, the operation controller, and the viewingdirection determiner. Alternatively, the robot 100 according to thedisclosure may include dedicated hardware, such as an applicationspecific integrated circuit (ASIC), a field programmable gate array(FPGA), or any of various control circuits, which functions as each ofthe target detector, the action detector, the operation controller, andthe viewing direction determiner, instead of the CPU. In this case, thefunctions of these components may be achieved by respective pieces ofhardware or may be comprehensively achieved by a single piece ofhardware. Alternatively, a part of the functions of the components maybe achieved by the dedicated hardware and the other part may be achievedby software or firmware.

The disclosure can provide the robot that preliminarily has aconfiguration to achieve the functions according to the disclosure. Thedisclosure can also provide a program that enables an existinginformation processing system, for example, to function as the robotaccording to the disclosure. That is, in a case in which the program forachieving the functional configurations of the robot 100 illustrated inthe embodiments is applied to and executed by, for example, the CPU forcontrolling the existing information processing system, the system canfunction as the robot according to the disclosure.

The program may be applied by any procedure. For example, the programmay be stored for application in a non-transitory computer-readablerecording medium, such as a flexible disk, a compact disc read-onlymemory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), ora memory card. Alternatively, the program may be superimposed on acarrier wave and applied via a communication medium, such as theInternet. For example, the program may be posted on a bulletin boardsystem (BBS) on a communication network and thus delivered. In thiscase, when activated and executed under the control of operating system(OS) as well as other application programs, the program may enable theabove processes to be executed.

The foregoing describes some example embodiments for explanatorypurposes. Although the foregoing discussion has presented specificembodiments, persons skilled in the art will recognize that changes maybe made in form and detail without departing from the broader spirit andscope of the invention. Accordingly, the specification and drawings areto be regarded in an illustrative sense rather than a restrictive sense.This detailed description, therefore, is not to be taken in a limitingsense, and the scope of the invention is defined only by the includedclaims, along with the full range of equivalents to which such claimsare entitled.

What is claimed is:
 1. A robot comprising: an operation unit that causesthe robot to operate; a viewing direction determiner that determineswhether a viewing direction of a predetermined target is toward therobot or not; and an operation controller that controls the operationunit based on a result of determination by the viewing directiondeterminer.
 2. The robot according to claim 1, further comprising: asolo operation determiner that determines whether the robot is executinga spontaneous solo operation or not, the solo operation being executedby the robot alone independently from the predetermined target in a casein which the predetermined target exists around the robot, the solooperation being not involving interaction with the predetermined target,wherein in a case in which the solo operation determiner determines thatthe robot is executing the solo operation, the viewing directiondeterminer determines whether the viewing direction of the predeterminedtarget is toward the robot or not.
 3. The robot according to claim 2,further comprising: an imager that captures an image of thepredetermined target, wherein the viewing direction determinerdetermines whether the viewing direction is toward the robot or notbased on the image of the predetermined target captured by the imager.4. The robot according to claim 2, wherein in a case in which the solooperation determiner determines that the robot is not executing the solooperation, the viewing direction determiner does not determine whetherthe viewing direction is toward the robot or not.
 5. The robot accordingto claim 2, wherein in a case in which the viewing direction determinerdetermines that the viewing direction is toward the robot, the operationcontroller causes the operation unit to execute a first operation forresponding to the viewing direction.
 6. The robot according to claim 5,wherein in a case in which the solo operation determiner determines thatthe robot is executing the solo operation, the viewing directiondeterminer determines whether the viewing direction is toward the robotat least for a predetermined duration or not, and in a case in which theviewing direction determiner determines that the viewing direction istoward the robot at least for the predetermined duration, the operationcontroller causes the operation unit to execute the first operation. 7.The robot according to claim 6, wherein the operation controller causesthe operation unit to execute the solo operation for the robot topretend not to notice the viewing direction until elapse of thepredetermined duration from when the viewing direction is turned towardthe robot.
 8. The robot according to claim 5, wherein the operation unitincludes a movable component, and the operation controller causes theoperation unit to execute the first operation by moving the movablecomponent.
 9. The robot according to claim 8, wherein the movablecomponent is a head or a tail of the robot, and the operation controllercauses the operation unit to execute the first operation by moving thehead or wagging the tail.
 10. The robot according to claim 5, whereinthe operation unit includes a display that displays an image, and theoperation controller causes the operation unit to execute the firstoperation by changing the image displayed on the display.
 11. The robotaccording to claim 5, wherein the operation unit includes a sound outputdevice that outputs a sound, and the operation controller causes theoperation unit to execute the first operation by outputting the soundthrough the sound output device.
 12. The robot according to claim 5,further comprising: an image recognizer that recognizes a face of thepredetermined target, wherein the operation controller causes theoperation unit to execute the first operation that varies depending on aresult of recognition by the image recognizer.
 13. The robot accordingto claim 5, wherein in a case in which the viewing direction determinerdetermines that the viewing direction is not toward the robot, theoperation controller causes the operation unit to execute a secondoperation different from the first operation.
 14. The robot according toclaim 2, further comprising: an action detector that detects an actionof the predetermined target to the robot, wherein in a case in which theaction detector detects the action, the operating unit executes aninteracting operation for responding to the action, and wherein in acase in which the action detector detects no action, the operating unitexecutes the solo operation after elapse of a predetermined intervalfrom last detection of the action of the predetermined target to therobot.
 15. The robot according to claim 1, wherein the predeterminedtarget is a human, an animal, or another robot different from the robot.16. The robot according to claim 2, wherein the solo operation isdefined based on information on emotions of the robot, information onsurrounding environment of the robot, or information on time.
 17. Therobot according to claim 16, wherein the information on the emotions ofthe robot is defined based on a combination of a first emotional valueand a second emotional value, the first emotional value representing alevel of calmness, the second emotional value representing a level ofexcitement, and the operation controller changes the emotions of therobot by varying the first emotional value and the second emotionalvalue in response to an action of the predetermined target during theinteraction with the predetermined target.
 18. The robot according toclaim 16, wherein the information on the surrounding environment of therobot is information on an outside of the robot except for thepredetermined target, the information including a sound volume, atemperature, an atmospheric pressure, a brightness, and existence of anobstacle.
 19. The robot according to claim 16, wherein the informationon the time includes a current time, a current date, and a currentseason.
 20. The robot according to claim 13, wherein the secondoperation is an operation for prompting the predetermined target to havethe viewing direction toward the robot.
 21. The robot according to claim20, wherein the second operation is rotation of a head of the robot ordisplacement of the robot.
 22. A method of controlling a robot,comprising: causing the robot to operate; determining whether a viewingdirection of a predetermined target is toward the robot or not; andcontrolling operation of the robot based on a result of determination inthe determining
 23. A non-transitory recording medium storing a programthereon, the program causing a computer of a robot to function as: anoperation unit that causes the robot to operate; a viewing directiondeterminer that determines whether a viewing direction of apredetermined target is toward the robot or not; and an operationcontroller that controls the operation unit based on a result ofdetermination by the viewing direction determiner.